Several recent science news items that I have discussed here and on my other blog, NeuroLogica, have a common thread weaving through them – the need for medical science to have high standards of rigor and to weed out both sloppy science and fraud at every level. The stakes for modern biomedical research are simply too high, and the costs of fraud and error too great.
Most dramatically there is the case of probable exposure of more than a decade of fraud in Alzheimer’s research. To quickly summarize this case, in 2006 neuroscientist Sylvain Lesné, published a paper in which he and his coauthors alleged to show that “A specific amyloid-β protein assembly in the brain impairs memory”. This had a large impact on Alzheimer’s (AD) research. One of the main competing theories for the ultimate cause of AD is that the brain degeneration results from the buildup of a toxic form of the amyloid-β protein. The controversy is over whether amyloid-β is simply a marker of AD or is it actually causing AD. This has significant implications for finding an effective treatment.
In his paper Lesné apparently showed that transferring amyloid-β to rats would impair their memory. This strongly suggests that the amyloid-β is causing the impairment. At the time this was taken as powerful support for the amyloid-β proponents, and certainly had an influence on the next decade and a half of AD research, including NIH and other funding.
However, a recent Science article and investigation details the accusations of another neuroscientist, Matthew Schrag, who claims to have found evidence of manipulated images in the Lesné paper, and in other Lesné research over the years. Images in scientific papers are often data, and manipulating images is therefore manipulating, or even manufacturing, data. It’s a giant no-no – it’s scientific fraud. So these allegations are extremely serious. This is why Science launched their own independent investigation, bringing in many experts in image analysis. Their conclusion is that they found “blatant” evidence of image manipulation. At present it seems that Lesné is solely responsible for the fraud as he was personally responsible for all images in his research, but his co-authors should have been more careful themselves.
Lesné’s original 2006 paper was published in Nature, who has not yet retracted the study, but did add this warning:
The editors of Nature have been alerted to concerns regarding some of the figures in this paper. Nature is investigating these concerns, and a further editorial response will follow as soon as possible. In the meantime, readers are advised to use caution when using results reported therein.
Unless they come to a very different conclusion from the Science investigators, I suspect a retraction is coming soon. This kind of fraud is devastating on many levels, and represents a massive waste of biomedical research resources. But if we want to look on the bright side it is also true that Lesné’s claims were met with initial skepticism (especially from those supporting competing theories of AD causality) and his findings were difficult to replicate. In the end the system did manage to discover the alleged fraud, mainly because the one thing you cannot fool with fraudulent research is reality itself. This case also reinforces why replication is so critical in scientific research.
This case also has implications for the underlying scientific question – is amyloid-β responsible for AD? This certainly undercuts the case for amyloid-β, but it does not mean that it is wrong. It can be true despite any fraud by Lesné. But it does put a controversial decision by the FDA which I discussed last year in a harsh light. In 2021 the FDA approved Aduhelm as a treatment for AD. The approval was controversial because it was based entirely on improving a marker for the disease rather than a traditional clinical outcome.
The FDA justified the approval by stating:
Patients receiving the treatment had significant dose-and time-dependent reduction of amyloid beta plaque, while patients in the control arm of the studies had no reduction of amyloid beta plaque.
The approval is contingent on later clinical trials showing clinical benefit. Ten of the 11 neuroscientists on the FDA advisory panel recommended not approving the drug, and the eleventh abstained. The FDA approved the drug anyway. Those neuroscientists understood that basing approval on a marker alone is highly problematic, and that amyloid-β as a marker for AD is still controversial. They likely feel now that the FDA has a huge “I told you so” coming.
In April of 2022 the US Centers for Medicaid and Medicare Services decided not to cover Aduhelm treatment, and many private insurers followed suit. Biogen, the maker of Aduhelm, has no choice but to proceed with their blinded clinical trials to prove efficacy – which is what should have happened in the beginning.
This entire saga also can be seen to support a recent initiative by the NIH to fund the development of educational modules for researcher, to teach high levels of scientific rigor. The modules would cover:
…biases in research; logical fallacies around causality; how to develop hypotheses; designing literature searches; identifying experimental variables; and reducing confounding variables in research.
These are all phenomena we discuss at length here at SBM, and they are indeed major problems in scientific research. It does seem that many researchers commit errors, such as p-hacking, simply because they are insufficiently aware of the problem and how to avoid it. It’s unclear if this kind of education would prevent fraud such as Lesné is alleged to have committed, but it wouldn’t hurt. It may make it easier for colleagues to detect fraud. And of course, there is no one solution to the problems of fraud and sloppy research, which is why we need filters in the system at every level.
The NIH’s efforts are definitely a step in the right direction. In fact, I would hope that not only do they make these educational materials available, but that funding eventually becomes dependent on certification in scientific rigor. The point of this is to reduce waste – waste of time, resources, and NIH funding. The Lesné affair shows how extreme the cost of bad research can be, estimated in the hundreds of millions of research dollars.
The institutions of science are imperfect, but strive to be self-corrective. While this is a sad and frustrating case, the response is appropriate – weed out the fraud, make corrections, and figure out how best to reduce similar fraud in the future. The NIH initiative is one method. But journals also need to add better filters prior to publication as well.
